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ON  GENERALIZATIONS  OF  COCHRAN'S  THEOREM 
AND  PROJECTION  MATRICES 


by 

Akimichi  Takemura 


1.  Introduction 

In  statistical  analysis  the  decomposition  of  a  sum  of  squares  into 
various  independent  components  is  essential  in  ANOVA-type  procedures. 

In  this  connection  Cochran's  theorem  gives  nice  conditions  to  guarantee 
the  independence  of  components.  There  have  been  many  generalizations 
or  refinements  of  Cochran's  theorem, and  recently  Anderson  and  Styan 
(1980)  reviewed  those  results  and  gave  new  ones. 

In  this  paper  we  present  a  unified  approach  to  those  problems  by 
consistent  use  of  "projection"  matrices  which  are  generally  not  ortho¬ 
gonal.  This  approach  enables  us  to  extend  various  results  to  much  greater 
general ity. 

In  section  2  we  review  the  properties  of  projection.  In  section  3 
we  prove  some  generalizations  of  Cochran's  theorem.  In  section  4  we 
discuss  decomposition  of  matrices  into  projections  to  their  elgenspaces 
and  its  relation  to  Cochran-type  decomposition. 

Matrices  may  be  considered  as  real  or  complex.  Proofs  are  given 
in  general  vector  space  terms, and  often  they  cover  both  cases.  In  section 
4  we  use  complex  matrices  explicitly. 
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2 


Projection  Matrix 

In  this  section  properties  of  projection  matrices  are  summarized. 

To  be  self-contained  brief  proofs  will  be  given. 

A  matrix  A  is  called  a  projection  matrix  if  it  is  idempotent,  i.e., 

2 

A  •  A  .  If  in  addition  A'  ■  A  (or  A*  ■  A  for  complex  matrices)  then 
A  is  called  an  orthogonal  projection  matrix^  Orthogonal  projections 
have  many  extra  properties  which  are  not  shared  by  general  projections. 
However,  there  are  quite  a  few  facts  which  have  counterparts  in  general 
projections  or  hold  without  orthogonality.  Moreover,  the  advantage  of 
developing  a  theory  for  general  projections  is  that  the  theory  holds  for 
vector  spaces  not  equipped  with  an  inner  product. 

We  need  some  definitions  about  subspaces  of  a  vector  space.  Let  X 
be  a  vector  space  and  U^,...,!^  be  subspaces  of  X  .  . are 

rk 

called  (linearly)  independent  if  Xj^  €  ,  i  ■  l,...,k  ,  x^  -  0 

implies  x^,  *  0  i  **  l,...,k  .  When  U^,...,U^  are  independent,  the  sub¬ 
space  U  spanned  by  is  called  the  direct  sum  of  U^'s  and 

_k 

denoted  by  U  *  U^+  . .  .+U^  *  £  ^  .  An  element  y  of  U  has  a  unique 

representation  y  ■  x^+. ..+  x^  where  x^  €  ,  i  ■  l,...,k  .  It  is 

easy  to  see  that  the  union  of  bases  of  U^’s  form  a  basis  of  U  and 
therefore  dim  U  »  J  dim  .  For  a  matrix  A  the  column  vector  space 
or  the  range  of  A  is  denoted  by  C(A)  .  The  null  space  of  A  is 
denoted  by  N(A)  .  X(A)  -  {X^,...,X^}  is  the  set  of  distinct  eigenvalues 
of  A  and  E(X^)  ■  N(A-XjI)  denotes  the  eigenspace  associated  with  XA  . 

Usually  "projection”  is  used  to  mean  orthogonal  projection  but  here  we 
use  it  for  general  projection. 
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Now  we  state  properties  of  projection  matrices. 

Proposition  2.1.  The  following  conditions  are  equivalent: 

(i)  A  is  a  projection. 

(ii)  I  -  A  is  a  projection. 

(iii)  C(A)  ■  (x:Ax  ■  x}  . 

■V  -V  ****  v 

(iv)  N(I-A)  -  C(A)  or  N(A)  -  C(I-A)  . 

(v)  C(A)  and  C(I-A)  are  independent. 

Proof.  We  show  that  (i)  is  equivalent  to  each  of  (ii)-(v)  . 

(ii):  (i)  «  A(I-A)  -  0  «  (ii)  • 

2 

(iii):  Let  V  -  {x:Ax-x}  .  Assume  A  -  A  .  If  x  fc  C(A) 

then  x  -  Ay  -  A2y  »  A(Ay)  -  Ax  .  Therefore  x  6  V  . 

If  x  €  V  then  x  -  Ax  fc  C(A)  .  Therefore  V  -  C(A) . 

Assume  C(A)  -  V.  Then  Ax  6  C(A)  *  V.  Hence 
A(Ax)  -  Ax  Vx  .  Therefore  A2  -  A  . 

(iv):  (iv)  **  (iii)  • 

(v) :  Necessity  is  proved  in  the  next  proposition.  Sufficiency 
is  proved  noting 

(A-A2)x  -  (I-A)Ax  -  0  ,  )ht  , 
and  (A-A2)x  fc  C(A)  ,  (I-A)Ax  6  C(I-A).  Q.E.D. 

Properties  (iii),  (iv),  (v)  show  that  A  projects  a  vector 

x  f.  C(A)  to  C(A)  along  a  line  parallel  to  C(J"A)  . 


Proposition  2.2.  Suppose  ^.....A^  are  projections  such  that 
AjAj  -  0  for  1  +  j  .  Then  (1)  A  ■  A^  +  . . .  +  A^  is  a  projection,  and 
(ii)  CCA^'s  are  independent  and  C(A)  -  * 

rjj 

Proof .  Let  x^  fe  ■  C(A^)  then  x^  «  6iXi  *  Suppose  x^  ■  0  . 

Then 

?  *  *1>  *  &4  ’  ij*J  '  ?J'  1  '  1*-"-k  • 

v*lc 

Therefore  C(Ai>'s  are  independent.  Ax  ■  A^x  ,  therefore 

(2.1)  C(A)  c  CCA^  . 

2 

On  the  other  hand  AA^x  “  A^x  ■  A^x  implies  C(A^)  C-  C(A)  ,  i  ■  l,...,k 
Therefore 

(2.2)  C(A)  lJMl  C(At)  . 

From  (2.1)  and  (2.2)  C(A)  -  ^  C(A±)  .  Q.E.D. 

The  converse  of  the  above  proposition  is  given  by  the  following 
proposition. 

Proposition  2.3.  Let  A  be  a  projection  onto  V  -  C(A)  .  Suppose 
V  »  V  where  s  are  given  independent  subspaces.  Then 

A  *  A^  where  A^'s  are  uniquely  determined  projections  such  that 

A±Aj  -  0  for  i  *  j  and  C(At)  -  V±  ,  i  -  1 . k  . 


Proof.  Let  VQ  -  C(I-A)  .  Then  V^V^...^  are  independent  and 

Vj  -  X  (whole  space).  Now  we  choose  vectors  ’  Jll*** 

x.  ,...,x.  }  such  that  {x.^-.-.x^  >  form  a  basis  for  . 

~iral  ~  laak  ~  i 
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Then 


form  a  basis  for  the  whole  space.  Let 


1“  ^01* 

be  a  matrix  with  column  vectors  x..  with  T.  "  (x. x.  ),  i  ■  0 

lj  ~  1  -11  ~lm . 

_1  *■ 
l,...,k  .  Let  T  be  partitioned  accordingly 


Then 


i  2 

Let  ■  T^T  .  Now  it  is  easy  to  check  that  A^  ■  A^  ,  CtA^)  ■  , 

rk. 

and  AjA^  »  0  for  i  4  j  .  To  prove  uniqueness  let  A  *  2.^  A^ 

•  ^  Bj  .  Then  (A^Bj)  x  -  0  ,  Vx  ,  implies  AjX  -  h±x  ,  i  •  1, 

... ,k  ,  because  (A^-B^)  x  fc  .  Hence  ,  i  ■  l,...,k  .  Q.E.D 

In  view  of  the  above  two  propositions  it  may  well  be  justified  to 
call  projection  matrices  A^  ,  i  ■  1, . . . ,k  ,  (linearly)  independent  if 
AjAj  ■  0  for  i  y  j  . 

We  note  here  the  relation  between  diagonalizable  matrices  and  pro¬ 
jections.  Let  A  be  a  p  x  p  diagonalizable  matrix  and  let  X(A) 

•  (Xx . Xk)  .  There  exists  a  nonsingular  T  such  that 

T~XAT  -  da 

where  is  a  diagonal  matrix  with  X^'s  as  diagonal  elements  in  an 

appropriate  order,  i.e., 

m  diag(X^, . . . .X^.Xj, . . . ,  X^)  . 
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Now  A  ■  TD^T~^  and  with  the  similar  partitioning  of  T  T~*  as  in 
the  proof  of  Proposition  2.3 

(2.3)  A  -  Xi  H± 

where  H^'s  are  independent  projections  onto  eigenspace  E(X.)'s  of  X 
It  follows  that  for  any  polynomial  P 

(2.4)  P(A)  -  l\ml  P(X±)  Ht  . 

rk 

H^'s  are  unique  by  Proposition  2.3  and  I  *  Conversely  if 

k  2 

E(X^)  -  X  then  A  can  be  diagonalized  by  T  as  in  the  proof  of 
Proposition  2.3. 

When  0  fc  X(A)  then  putting  X^  *  0  we  can  write  (2.3)  as 

*  -  A!  Si  • 

(2.4)  holds  with  k  replaced  by  k  -1  if  P(0)  ■  0  ,  i.e.,  P  does  not 
have  constant  term. 

Despite  its  apparent  plausibility  we  need  the  diagonalizability  of 

a  projection  matrix  to  prove: 

2 

Proposition  2.4.  If  A  ■  A  then  rank  A  *  tr  A  . 

2 

Proof.  A  ■  A  Implies  X(A)  c  {0,1}  .  It  will  be  shown  that  A 
is  diagonal izable  in  section  4  and  therefore 

rank  A  ■  number  of  eignevalues  1 
■  tr  A  .  Q.E.D. 


Eigenspaces 


E^)  ’s 


are  always  independent. 


i 


3. 


i 


Cochran-Type  Theorems 

In  terms  of  matrix  theory  Cochran's  theorem  essentially  asserts  the 

rk. 

following:  Let  A  be  an  orthogonal  projection  and  A=£*_^  A^  where 
s  are  symmetric.  If  rank  A  ■  rank  A^  then  A^'s  are  inde¬ 

pendent  orthogonal  projections.  Various  kinds  of  extensions  and  refine¬ 
ments  are  discussed  in  Anderson  and  Styan  (1980).  In  this  section  we 
present  simpler  proofs  for  a  number  of  them  and  give  a  generalization 
(Theorem  3.3  together  with  Corollary  4.2). 

We  begin  by  showing  that  Cochran's  theorem  holds  without  orthogonality. 

rk 

Theorem  3.1.  (Cochran's  theorem)  Let  A  =  2.^^  A.^  .  A^'s  are 
independent  projections  if  and  only  if 

(i)  A  is  a  projection, 

(il)  rank  A  *■  Y^  rank  A.  . 

~  ^1=1  ~i 

The  key  point  to  prove  this  is  contained  in 

rk 

Lemma  3.1.  Let  A  *  2.^^  A^  where  A^'s  are  general  matrices.  Then 
(i)  C(A^)'s  are  independent  and  C(A)  *  C(A^) 

if  and  only  if 

(ii)  rank  A  -  rank  A^  . 

Proof. 

Necessity:  dim{C(A)}  -  dim{C(Ai)}.  This  is  equivalent  to  rank  A 


Sufficiency:  We  show  the  independence  of  C(A^)  by  contradiction. 
Suppose  that  there  are  vectors  x1,...,xk  with  at  least  one  nonzero 
vector  such  that  x^CfAj),  i  -  ,  and  xi  »  0  .  Without 

loss  of  generality  let 

2  *  *1  =  "Ii=2  • 

Let  {x^,q^, . . . , q^}  be  a  basis  for  C(A^)  where  m  =  rank  A^  -  1  . 

Then  C(A)  c  C^, . . .  .A^  =  C^, . . . ,  q^A^  . . .  .A^  .  But  dim{C(A)  } 

■  rank  A  and 

dim  CKq^,...,^,  , . . . ,  A^)  £  m  +  dim  C(A^)  =  rank  A  -  1 

which  is  a  contradiction.  Therefore  C(Ai)'s  are  independent.  Now 
rk 

C(A)  c  l±ml  C(A±)  but  again  by  the  rank  condition  (ii)  we  get  C(A) 

-  C(.A±)  .  Q.E.D. 

Proof  of  Theorem  3.1. 

Sufficiency:  From  the  lemma  we  know  C(Aj)  c.  C(A)  ,  i  =  l,...,k  . 

Therefore  by  (iii)  of  Proposition  2.1  we  get  AA^x  *  A^x  .  Then 


o 

By  Independence  of  C(A^)'s  we  get  A^A^x  =0  i  ^  j  and  (A^-A^Jx  * 

2 

This  being  true  for  any  x  shows  A^A^  *  0  ,  i  ^  j  ,  and  A^  *  A^  . 

Necessity:  In  view  of  lemma  3.1  necessity  was  proved  already  in 
Proposition  2.2.  Q.E.D. 

Chipman  and  Rao  (1964),  Khatri  (1968)  give  more  algebraic  proofs  of 


this  theorem. 


i  o 


I 


He  proceed  to  generalize  this  result  to  an  r-potent  matrix.  A  matrix 
A  is  called  r-potent  if  Ar  ■  A  .  A  general  theorem  is  given  by  Anderson 
and  Styan  (1980,  their  theorem  3.3).  Here  we  present  a  simpler  (but  equi¬ 
valent)  version. 

Theorem  3.2.  (r-potent  matrix)  Let  A  ■  A^  and  r  3  . 

( ~  i  =  £i  *  £i~  j  9  ’  1  *  j  » 

if  and  only  if 

(i’)  Ar  *=  A  ,  (ii')  rank  A  =  rank  A^  , 

(iii’)  AAi  =  A±A  ,  i  -  l,...,k  . 

Proof. 

r-2 

Necessity:  We  need  to  prove  (ii').  By  multiplying  (i)  by  A^  or 
Aj-2  we  get  A2(r_1)  -  A^"1  ,  A*-1  A*-1  =  0  ,  i  4  j  .  Therefore  the 

previous  theorem  applies  to  A^  ^  and  we  get 

rank  Ar  1  -  rank  A*  1  . 

—1 

But  (i)  and  (i1)  show  that  rank  Ar  =  rank  A  ,  rank  ■  rank  A^ 

noting  rank  A  _>  rank  Ar_1  >  rank  Ar  «  rank  A  ,  etc.  (ii')  follows  from 
this. 


Sufficiency:  From  (iii')  we  get 


Hence 


By  Lemma  3.1  C(A^)'s  are  independent  and  therefore  A^A^x  *  0  ,  Vx,  j  f  i 

Hence 


=  5  *  i  *  i  • 

Now 

0  =  Ar  -  A  -  Ii=1(A^-A.)  . 

Noting  (A^-A^)x  fc  CKA^),  \x,  we  get 

Aj  =  A.  ,  i  =  1, . . . ,k  .  Q.E.D. 

~i  ~i 

We  have  assumed  r  3  above  because  (iii')  is  not  needed  in  the 

case  r  *  2  .  Anderson  and  Styan  (1980,  their  Theorem  3.3)  replaces 
r-2  r-2 

(iii')  by  A^A  =  A  A^  ,  for  example.  This  condition  is  nice  in  the 
sense  that  it  reduces  to  the  null  condition  when  r=»2.  On  the  other  hand 
(iii')  is  desirable  in  connection  with  simultaneous  diagonalizability 
(Theorem  4.3  in  section  4)  and  actually  easier  to  check  for  r  >  3  . 

Our  proof  of  sufficiency  is  quite  general  and  it  yields  the  follow¬ 
ing  result  almost  without  changes. 

Theorem  3.3.  Let  P(x)  be  any  polynomial  of  deg  P  ^  2  .  Suppose 

(i)  P( A)  -  0  , 

p  k 

(ii)  rank  A  =  rank  A^  , 

(iii)  AA^  ■  A^A  . 
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jLl.  -  -ft &  •_ -sai 


Then 


if  P(0)  -  0 


PCA^  =  0,  i  -  1 . k  , 


=  0  and 


lA1P(Ai)  =  0,  i=l,...,k,  if  P(0)  4  0 


Proof.  As  in  the  proof  of  Theorem  3.2  we  get  A^A^ 


0,  i  i  j 


Therefore  0  =  P(A)  -  l£=1p(Ai>  if  P(0)  =  0 


Then 


0  -  llml  A±  Q(A±) 


where  Q(x)  =  P(x)/x  .  By  independence  of  C(A^)'s  we  get 
Ai  =0»  i  =  1, . . . ,k  .  Therefore  P(Ai>  *  Ai  Q(Ai)  “  0  .  If 

P(0)  ^  0  ,  let  R(x)  =  xP(x)  .  Then  R(0)  =  0,  R(A)  *  0  ,  and  by 

the  above  argument  R(A^)  =  A^  PCA^)  =  0^  i  “  1 . k  .  Q.E.D. 

When  P(0)  =  0  the  converse  of  this  is  true  except  tor  ^ii;. 

A  partial  converse  will  be  given  in  Corollary  4.2. 


4.  Decomposition  to  Projections 

The  decomposition  discussed  in  the  previous  section  may  be  called  the 
Cochran-type  decomposition.  Here  we  want  to  investigate  another  type 
of  decomposition  which  we  call  the  decomposition  to  projections'?  For  a 

4 

diagonalizable  matrix  such  decomposition  was  discussed  at  the  end  of 

o 

It  is  often  called  "spectral  decomposition,  as  in  the  case  of  orthogonal 
projections. 

^A  more  "mathematical"  term  is  "seraisimple" . 


section  1.  First  we  derive  a  simple  criterion  which  guarantees  the 

diagonal izability  of  a  matrix.  After  that  the  relation  of  two  types 

of  the  above  decompositions  will  be  discussed  and  the  main  theorem  of 

this  section  (Theorem  4.4)  asserts  that  two  types  of  projections  commute. 

Theorem  4.4  will  be  applied  to  generalize  Cochran's  theorem  to  sum  of 
2 

weighted  X  variables. 

To  discuss  the  diagonalizability  of  matrices  we  rely  on  the  theory 

of  Jordan  canonical  form  and  minimal  polynomial.  For  a  p  *  p  matrix  A 

let  A(A)  ■  {Aj,...,A^}  and  let  m^  be  the  multiplicity  of  eigenvalue 

A^  ,  i  *  l,...,k  =  p)  •  E(A  )  •  N(A-A^I)  is  called  the 

m . 

eigenspace  associated  with  A^  .  E  (A^)  =  NCEA-A^I]  )  is  called  the 
wide-sense  eigenspace  associated  with  A_,  .  A  polynomial  <!>.(•)  is 

1  A 

called  the  minimal  polynomial  of  A  if  4>A  is  the  lowest  degree  poly¬ 
nomial  such  that  <{>a(A)  =  0  .  The  theory  of  Jordan  canonical  forms  tells 
us  the  following:  E  (X^)  9  i  *  l,...,k,  are  independent,  dim  E  (X^)  •  » 

and  EW(A^)  ■  X  (the  whole  space) ;  there  exists  a  nonsingular  T  =  (T^, 

w 

....T^)  with  T^:  p  x  consisting  of  a  basis  of  E  (A^)  such  that 


f 


Si 


o 


0 


0 


T-1AT  => 


0 


0 


0 


where  is  an  m^  *  m^  upper  triangular  matrix;  furthermore  is  of 

the  form 
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matrix  which  has  the  form 


where  is  a  8^  *  8^ 

A1  1  0  •••0 

0  Af  1  •••0 

•  •  •  • 

•  •  •  * 

•  •  •  • 

0  0  0  •••  1 
0  0  0  •••  A 

The  minimal  polynomial  4>  turns  out  to  be 

A 

k  h 

(4.1)  <t>A(x)  -  nJ=1(x-A1)  1 

where  8^  ■  max^lS^}  .  See  Shilov  (1971,  Chapter  6)  for  example. 

If  8.  m  1  then  C.  “  A.  I  .  Therefore  we  have 
i  ~i  i  -m^ 

Fact  3.1. 

The  following  conditions  are  equivalent: 

(i)  A  is  diagonalizable. 

(ii)  -  1  ,  i  -  l,...,k  . 

( iii)  E(At)  -  Ew(At)  ,  i  -  l,...,k  . 

(iv)  "  X  . 

These  conditions  are  rather  hard  to  check  for  particular  matrices. 
Therefore  it  is  desirable  to  have  a  sufficient  condition  for  the  diagonal! 
zability.  Here  is  a  strong  but  fairly  general  sufficient  condition. 


Lemma  4.1.  Let  P(x)  (P^O)  be  a  polynomial  such  that  P(x)  ■  0 
has  no  multiple  root.  Suppose  P(A)  -  0  ;  then  A  is  diagonalizable. 
Proof .  Let 

P(x)  *  Q(x)  4>a(x)  +  R(x) 

where  deg  R  <  deg  <J>A  .  Then  P(A)  *  Q(A)  <j>A(A)  +  R(A)  and  we  have 

R(A)  *  0  .  But  minimality  of  <{> .  implies  R(x)  =  0  .  Therefore 
~  '  k  6i  ~ 

P(x)  ■  Q(x)  ni_1(x-A1)  by  (4.1).  By  assumption  about  P  we  get 
3^  ■  1  >  i  *  l,...,k,  and  P  is  diagonalizable.  Q.E.D. 

Applying  this  criterion  we  obtain  the  following  theorem. 

Theorem  4,1.  (Decomposition  to  Projections)  Let  P(x)  (P£0)  be 
a  polynomial  such  that  P(x)  *  0  has  no  multiple  root.  Suppose  P(A) 

-  0  >  then  A  is  diagonalizable  and 

(4.2)  A  -  A.  H,  +  ...  +  A,  H. 

~  1  ~1  k  ~k 

where  A(A)  ■  {A^,...,A^}  ,  H^'s  are  independent  projections  and 
2.^^  ■  I  .  This  expression  is  unique. 

Proof .  By  Lemma  4.1  A  is  diagonalizable.  Decomposition  (4.2)  is 
given  in  (2.3).  Q.E.D. 

Corollary  4.1.  Let  A  be  r-potent,  i.e.,  Ar  -  A  .  Then 


AW*' 
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(4.3) 


A  •  Hx  +  u«2  +  . . .  +  a)  Hj._1 


H,  +  uM,,  +  . . .  +  u>  H  .  +  0  •  H 
-1  ~2  ~r-l  ~r 


where  oi  *  ei2ir/(r  1)  an(j  n^»s  are  independent  projections.  This 
expression  is  unique. 

Proof .  Let  P(x)  ■  xr  -  x  ■  x(x-w)  ...  (x-wr-2) (x-1) .  Then  P(x)  has 
no  multiple  root.  Noting  that  X(A)  c  (l,u), . . .  ,wr-2,0}  (-4.3)  follows 
from  (4.2).  Q.E.D. 

This  decomposition  has  the  following  intuitive  meaning.  First  note 
C(A)  »  C(wi_1Hi)  -  C(Ht)  .  Now  Ar  -  A  implies  C(A)  -  C(Ar) 

•  C(A2r_1)  -  ...  .  On  the  other  hand,  C(A)  ^  C(A2)  3  ...  ,  therefore, 

C(A)  -  C(aS  ,  k  -  2,3,...  .  Let  x  fc  C(A)  then  x  -  Ay  .  Then 

Ar  1  x  ■  Ary  “Ay  *  x  .  Therefore,  A  can  be  interpreted  as  "rotating" 

C(A)  completely  in  r  -  1  steps.  Actually  (4.3)  tells  us  that  A 
divides  C(A)  into  r  -  1  independent  spaces  C(Hj)  ,  i  *  l,...,r  -  1  , 
and  rotates  C(Hj)  by  the  angle  (i-l)2ir/(r-l)  . 

At  this  point  we  have  two  decompositions:  Cochran-type  decomposition 
(Theorem  3,3)  and  decomposition  to  projections  (Theorem  4.1).  It  is  con¬ 
ceivable  that  these  two  decompositions  are  "independent"  and  the  order 
of  applying  them  is  interchangeable. 

Theorem  4.2.  (Commutativity) 

Let  P(x)  (PJfO)  be  a  polynomial  with  no  multiple  root.  Let 
A  *  Aa  .  Suppose 


(i)  P(A)  -  0  , 

(ii)  rank  A  -  rank  A1  , 

(iii)  AAf  -  AaA  i  -  l,...,k  . 

Than  there  exist  independent  projections  Hn . H12,... 

H^i  •  •  •  such  thst 

^1  “  X1  Sll  +  A2  Sl2  +  *•*  +  Xi  Sli  * 

^2  "  X1  ?21  +  X2  ?22  +  •**  +  Xi  ?2£  ' 

<4. A)  : 

Sk  “  X1  Ski  +  X2  Sk2  +  ‘ ‘ ‘  +  Xi  Ski  » 

A  “  A!  Si  +  X2  S2  +  *•*  +  Xi  Si 

where  Hj  ■  ^i»i  Sij  an<*  ■  A(A)  -  {0}  .  These  expressions 

are  unique. 

We  first  prove  the  following  essential 

Theorem  4.3.  (Simultaneous  Diagonal izat ion) 

Let  A,B  be  diagonal izable.  Then 

AB  -  BA 

if  and  only  if  A,B  are  simultaneously  diagonal izable. 
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- .... 


M- 


IX 


Proof. 


Sufficiency:  Let  A  -  TD^T-1  ,  B  -  TD^T-^  ,  then  AB  -  TD^T”1 
■  TD^T-1  ■  BA  . 

Necessity:  Let  X(A)  -  {X^...,!^  ,  X(B)  »  and  A  -  £ 

X±  ,  B  *  Uj  H®  .  Suppose  x  t  E^)  -  C(H^)  .  Then  ABx  -  BAx 

-  X^x  .  Therefore  Bx  fc  E(X1>  or  C(BH^)cc(H^)  .  Noting  I  -  H* 

we  get 

0  «  B  -  B  -  IJ.jWJb-BhJ)  . 

Therefore  H^B  ■  BH^  .  By  Corollary  4.1  H*  is  diagonal treble  ,  and 
repeating  the  same  argument  we  obtain 


H®  -  H®  H*  for  any  i  and  j  . 

Let  H..  -  H*  H®  .  Then 
-  1J  ~ 1  ~  J 


H*  H®  H*  H® 


(H^)2  <H®)2  -  H*  H®  -  ; 


similarly 


®ij  5iy"  ?  for  1  *  *'  or  J  • 

It  follows  that  Hy  ,  i  -  l,...,k  ,  j  ■  1, are 

independent  projections  such  that  I  ■£.  .  H  .  By  Cochran's  theorem 

-  IfJ  ^lj 

J.  .  rank  H..  "  p  .  Now  taking  appropriate  bases  from  C(H..)'b  and 
*»3  ~XJ 

letting  T  as  in  the  proof  of  Proposition  2.3,  it  is  easy  to  see 
that  A,B  are  simultaneously  diagonalized  by  T  and  A, B  can  be 
written  as 


iitttAiiifcnriritm*  ■  dnrigjfr*"- 
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(4.5) 


t  “  ^i-1  *i  Ej«i  ?ij 


?  -  lU  UJ  lh 


Now  C(Hy)  c  C(H*)  nc(H®)  -  E(At)  n  E(Pj)  .  On  the  other  hand  if 
x  €  E(Aj)  fl  E(Mj)  then  H^x  "  HjX  -  H^x  -  x  k  C(H^)  •  Therefore 

C(Ht  )  -  K^)  H  E(Pj)  .  Q.E.D. 

Remark:  It  is  immediate  to  generalize  the  above  proof  to  more 
than  two  dlagonalizable  matrices  commuting  with  each  other.  For  more 
standard  proofs  see  Satake  (1975,  Sec.  4.2,  ex.  1),  Greub  (1975,  Sec. 
13.27). 

Now  we  are  in  a  position  to  prove  Theorem  4.2. 

Proof  of  Theorem  4.2.  The  case  deg  P  *  1  reduces  to  Cochran's 
Theorem.  Suppose  deg  P  ^  2  ,  then  by  Theorem  3.3  •  0  ,  i  ^  j,  and 

P(A. )  *0  or  A.  P(A.)  ■  0  .  By  Lemma  4.1  A. 's  can  be  diagonalized. 

Let  ^  *  X(Aj)  ~  •  Me  want  to  show 

A(A±)  -  {0}  C  A(A)  -  {0} 

and 


E(M1:J)  C  E(Xa) 

where 

Aa  '  “ij  • 

Suppose  x  k  E(utj)  .  Then 

;  ■  <V2  ?  ■ 

Dividing  by  p^  we  get  Ax  -  p^  x  }  hence  py  k  X(A)  -  {0}  and  if 
Mjj  •  *a  then  E(py)  c  E(pa)  . 
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Summarizing  this  we  obtain  now  from  Theorem  4.2 


A 

h 

*k 


X^  ■+■  •  •  •  ^  X4  ) 

xi  5n  +  +  x«.  5u  • 

X1  5kl  +  ‘ "  +  xi  3kJt  * 


where 


(i)  H  , are  independent  projections, 

(ii)  for  fixed  i  H  are  independent  projections,  and 

(iii)  E(piQ)  “  c  c(Hj)  “  E^j^*  1  *  1»****^»  where 

Wj  “  A  . 
id  j 


Now 


A  x  “  ~j  ~  * 

Ii.,  *i  ;  ■  Si-1  Si-1  5,J 


X  . 


Therefore 


<V^i-l~ij)Aj  *  "  ¥?> 


and  ve  get 


<*.»)  ?j  -  ii_,  ?u  ■ 

Ut  H  -  lJ.j  Hj  •  ij-i  "u  •  »°« 

rank  A  •  rank  Hj  ■  rank  H  , 

-i 

rank  At  -  rank  . 
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Hence 


Implies 


£  rank  ■  rank  A 


yy.  .  rank  H..  ■  rank  H  . 

LLUi  ~ij 


Therefore  by  Cochran's  Theorem  (Theorem  3.1)  H^'s  are  independent 
projections.  From  the  construction  the  expressions  are  unique.  Q.E.D. 

Corollary  4.2.  (Converse  to  Theorem  3.3)  Suppose  P(x)  has  no 

k 

multiple  root  and  P(0)  =  0  .  Let  A  =  J,  _  At  .  If 

(i)  *i  Aj  -  0  ,  i  *  j  , 

(ii)  P(Ai)  -  0  ,  i  -  l,...,k  , 

then 

(iii)  P(A)  -  0  , 

(iv)  A  Aj  *  ^  A  , 
rk 

(v)  rank  A  ”  rank  A^  . 


Proof .  (iii),  (iv)  follow  immediately  from  (i)  and  (ii).  There¬ 
fore  we  assume  (i)  -  (iv)  instead.  Then  the  above  proof  requires  no 
change  up  to  (4.6).  Now  we  want  to  show  that 


H.  H, 
-is  ~jt 


0 


whenever  i  4  j  or 
•  j*  t  .  Let  ct,0 
then  let  E(Vla)  * 


i  t  without  the  rank  condition.  Fix  s,t  such  that 


be  such  that  yia  -  Ag  , 


At  .  If  Ag  t  A(A1)-{0) 


0  instead.  In  any  case  E(ylQ)  0  E(u1g)  c  E(Ag) 


fi  E(Xt)  *  0  .  Theorem  4.3  applied  to  A^,A^  (note  A^  *  Aj  *  0) 
implies  that  in  (4.5)  the  second  sums  are  just  single  terms  and  now  s  i*  t 
Implies  His  Hjt  -  0  .  Then  for  i  +  j  ,  0  —  Af  Aj  -  Ag  Hig  . 

Noting  C(H.  H  )  c  C(H  )  we  get  H.  H.  ■  0  .  Therefore  we  have 

■'••IS  ~J  S  ~  S  IS  8 

shown  Hy's  are  independent  and  by  Cochran's  Theorem  used  in  the  other 
direction  we  get  rank  H  ■  rank  which  yields  rank  A  * 

rank  A^.  Q.E.D. 

Khatri  (1977,  Sec.  3)  discusses  similar  results  without  assuming  the 
commutativity  (iii)  which  is  essential  for  our  development. 

Actually  we  can  put  Theorem  4.2  into  a  more  natural  and  general  form. 

Theorem  4.4.  (Commutativity)  Let  A  *  A^  •  Suppose 


(i)  A  is  diagonal izable, 

(ii)  rank  A  =  rank  At  , 

(iii)  A  Aj  *  Aj  A  i  =  l,...,k  , 

Then  A^'s  are  simultaneously  diagonal Izable  and  (4.4)  holds. 

Proof .  By  Fact  4.1,  A  is  diagonalizable  **  <(> .  has  no  multiple  root. 
Therefore  the  result  follows  if  we  let  P  **  <p^  in  Theorem  4.2  .  Q.E.D. 

Now  looking  at  (4.4)  we  see  that  row-wise  it  gives  a  decomposition 
to  projections  and  column-wise  it  gives  Cochran-type  decompositions. 

In  addition,  marginal  sums  add  up  correctly.  To  get  we  could  either 

(1)  decompose  A.  into  projections,  or  (2)  decompose  A  into  projections 
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H. , . . . ,H.  and  let  H..  ■  A,  H.  .  In  this  sense  the  two  decompositions  commute. 
~x,  1 

2 

Finally  we  give  an  application  of  Theorem  4.4  to  weighted  y  -variables. 

If  A  and  A^'s  are  real  symmetric  the  theory  developed  in  this  section 
can  be  interpreted  entirely  in  real  numbers  and  real  matrices.  Further¬ 
more,  diagonalization  can  be  restricted  to  the  ones  by  orthogonal  matrices, 
which  is  always  possible.  In  statistical  terms,  then.  Theorem  4.4  reads: 


Theorem  4.5.  Let  x^,...,x  be  independent  standard  normal  variables. 
Let  Q*  x:Ax,  =  x' Aj  x, . . .  ,Qk  = 
such  that 

<»  6  ’  Ew  • 

(ii)  rank  A  «*  1^=2  ran^  * 

(iii)  AAt  -  A±A  ,  i  -  l,...,k  . 


x*A^  x  be  quadratic  forms  in  x^'s 


Then  there  exist  independent  standard  normal  variables 
that  (y^,...,yp)'  »  T(x^, . . . .x^) '  for  some  orthogonal 


and 


such 


Q 

^2 


X1  y?  +  X2  y2  +  •••  +  \n  ym 


1.  y^  +  ...  +  1  y^ 
11 


x  -  y«  +  •••  +  K  _  y. 


n^+1  Jm^+1 


m1+m2  ^m1+m2 


i  2  ,2 

Di-Di^+1  ym-m^+2  •  •  •  +  m  ym 
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rank  A. 


m  ■»  rank  A  . 


form  is  essentially  proved  in  Luther  (1965,  his  Theorem  1).  See  also 
Khatri  (1977)  for  further  discussion. 
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